Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement light-client spec tests #4908

Merged
merged 13 commits into from
Dec 19, 2022
Merged

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 13, 2022

Motivation

Implement light-client spec tests from https://github.com/ethereum/consensus-specs/tree/dev/tests/formats/light_client

Description

  • single_merkle_proof
  • sync: Re-used new code in actual light-client class
  • update_ranking

@github-actions
Copy link
Contributor

github-actions bot commented Dec 13, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: d4b1bad Previous: 2ece180 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.2336 ms/op 1.1539 ms/op 1.07
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 72.245 us/op 67.962 us/op 1.06
BLS verify - blst-native 1.8691 ms/op 1.8708 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 3.8186 ms/op 3.8168 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 8.2065 ms/op 8.2030 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 29.720 ms/op 29.699 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 39.356 us/op 39.331 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 153.81 us/op 153.56 us/op 1.00
getAttestationsForBlock 90.695 ms/op 86.234 ms/op 1.05
isKnown best case - 1 super set check 430.00 ns/op 418.00 ns/op 1.03
isKnown normal case - 2 super set checks 411.00 ns/op 407.00 ns/op 1.01
isKnown worse case - 16 super set checks 412.00 ns/op 409.00 ns/op 1.01
CheckpointStateCache - add get delete 8.6430 us/op 8.0350 us/op 1.08
validate gossip signedAggregateAndProof - struct 4.2761 ms/op 4.2710 ms/op 1.00
validate gossip attestation - struct 2.0338 ms/op 2.0269 ms/op 1.00
pickEth1Vote - no votes 2.1200 ms/op 2.1299 ms/op 1.00
pickEth1Vote - max votes 16.889 ms/op 16.536 ms/op 1.02
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.885 ms/op 12.042 ms/op 0.99
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.391 ms/op 21.062 ms/op 1.02
pickEth1Vote - Eth1Data fastSerialize value x2048 1.1977 ms/op 1.1914 ms/op 1.01
pickEth1Vote - Eth1Data fastSerialize tree x2048 9.5678 ms/op 9.9918 ms/op 0.96
bytes32 toHexString 1.0380 us/op 1.0100 us/op 1.03
bytes32 Buffer.toString(hex) 573.00 ns/op 554.00 ns/op 1.03
bytes32 Buffer.toString(hex) from Uint8Array 929.00 ns/op 921.00 ns/op 1.01
bytes32 Buffer.toString(hex) + 0x 576.00 ns/op 564.00 ns/op 1.02
Object access 1 prop 0.27800 ns/op 0.28000 ns/op 0.99
Map access 1 prop 0.25600 ns/op 0.25800 ns/op 0.99
Object get x1000 16.534 ns/op 16.007 ns/op 1.03
Map get x1000 0.99400 ns/op 0.99500 ns/op 1.00
Object set x1000 115.70 ns/op 109.46 ns/op 1.06
Map set x1000 71.639 ns/op 66.500 ns/op 1.08
Return object 10000 times 0.37370 ns/op 0.37420 ns/op 1.00
Throw Error 10000 times 6.8588 us/op 6.7567 us/op 1.02
fastMsgIdFn sha256 / 200 bytes 4.3360 us/op 4.3520 us/op 1.00
fastMsgIdFn h32 xxhash / 200 bytes 459.00 ns/op 440.00 ns/op 1.04
fastMsgIdFn h64 xxhash / 200 bytes 633.00 ns/op 631.00 ns/op 1.00
fastMsgIdFn sha256 / 1000 bytes 13.537 us/op 13.596 us/op 1.00
fastMsgIdFn h32 xxhash / 1000 bytes 605.00 ns/op 631.00 ns/op 0.96
fastMsgIdFn h64 xxhash / 1000 bytes 715.00 ns/op 700.00 ns/op 1.02
fastMsgIdFn sha256 / 10000 bytes 118.02 us/op 116.94 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 2.3380 us/op 2.3190 us/op 1.01
fastMsgIdFn h64 xxhash / 10000 bytes 1.6680 us/op 1.6720 us/op 1.00
enrSubnets - fastDeserialize 64 bits 2.5150 us/op 2.1610 us/op 1.16
enrSubnets - ssz BitVector 64 bits 886.00 ns/op 760.00 ns/op 1.17
enrSubnets - fastDeserialize 4 bits 293.00 ns/op 279.00 ns/op 1.05
enrSubnets - ssz BitVector 4 bits 819.00 ns/op 777.00 ns/op 1.05
prioritizePeers score -10:0 att 32-0.1 sync 2-0 99.029 us/op 92.180 us/op 1.07
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 137.13 us/op 136.00 us/op 1.01
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 271.34 us/op 245.06 us/op 1.11
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 405.27 us/op 400.89 us/op 1.01
prioritizePeers score 0:0 att 64-1 sync 4-1 518.55 us/op 497.06 us/op 1.04
array of 16000 items push then shift 2.8178 us/op 2.8011 us/op 1.01
LinkedList of 16000 items push then shift 19.256 ns/op 17.260 ns/op 1.12
array of 16000 items push then pop 219.93 ns/op 211.99 ns/op 1.04
LinkedList of 16000 items push then pop 17.704 ns/op 16.528 ns/op 1.07
array of 24000 items push then shift 3.9712 us/op 3.9667 us/op 1.00
LinkedList of 24000 items push then shift 22.487 ns/op 19.886 ns/op 1.13
array of 24000 items push then pop 210.90 ns/op 212.41 ns/op 0.99
LinkedList of 24000 items push then pop 19.873 ns/op 17.829 ns/op 1.11
intersect bitArray bitLen 8 29.345 ns/op 29.410 ns/op 1.00
intersect array and set length 8 162.45 ns/op 155.05 ns/op 1.05
intersect bitArray bitLen 128 126.06 ns/op 108.08 ns/op 1.17
intersect array and set length 128 2.3203 us/op 2.2863 us/op 1.01
Buffer.concat 32 items 5.9930 us/op 5.1750 us/op 1.16
Uint8Array.set 32 items 4.1950 us/op 4.1410 us/op 1.01
pass gossip attestations to forkchoice per slot 4.2883 ms/op 6.2763 ms/op 0.68
computeDeltas 5.9109 ms/op 5.6342 ms/op 1.05
computeProposerBoostScoreFromBalances 2.8510 ms/op 2.8365 ms/op 1.01
altair processAttestation - 250000 vs - 7PWei normalcase 3.9941 ms/op 3.5855 ms/op 1.11
altair processAttestation - 250000 vs - 7PWei worstcase 7.2532 ms/op 7.0176 ms/op 1.03
altair processAttestation - setStatus - 1/6 committees join 206.98 us/op 205.04 us/op 1.01
altair processAttestation - setStatus - 1/3 committees join 396.13 us/op 392.20 us/op 1.01
altair processAttestation - setStatus - 1/2 committees join 550.93 us/op 548.25 us/op 1.00
altair processAttestation - setStatus - 2/3 committees join 711.21 us/op 706.63 us/op 1.01
altair processAttestation - setStatus - 4/5 committees join 993.66 us/op 987.82 us/op 1.01
altair processAttestation - setStatus - 100% committees join 1.1758 ms/op 1.1623 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase 27.487 ms/op 25.769 ms/op 1.07
altair processBlock - 250000 vs - 7PWei normalcase hashState 42.089 ms/op 39.204 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase 82.013 ms/op 78.656 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase hashState 114.60 ms/op 106.78 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei normalcase 3.5285 ms/op 3.4938 ms/op 1.01
phase0 processBlock - 250000 vs - 7PWei worstcase 46.668 ms/op 45.026 ms/op 1.04
altair processEth1Data - 250000 vs - 7PWei normalcase 788.30 us/op 729.53 us/op 1.08
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 11.837 us/op 9.8880 us/op 1.20
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 29.454 us/op 28.440 us/op 1.04
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 17.746 us/op 11.628 us/op 1.53
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 12.924 us/op 8.9710 us/op 1.44
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 139.65 us/op 99.359 us/op 1.41
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 1.1045 ms/op 1.0066 ms/op 1.10
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.4989 ms/op 1.3440 ms/op 1.12
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.3673 ms/op 1.3056 ms/op 1.05
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 4.3527 ms/op 4.3179 ms/op 1.01
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 2.8205 ms/op 2.8428 ms/op 0.99
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 7.0593 ms/op 6.9202 ms/op 1.02
Tree 40 250000 create 702.24 ms/op 629.17 ms/op 1.12
Tree 40 250000 get(125000) 297.85 ns/op 282.10 ns/op 1.06
Tree 40 250000 set(125000) 2.2206 us/op 2.0914 us/op 1.06
Tree 40 250000 toArray() 33.185 ms/op 31.242 ms/op 1.06
Tree 40 250000 iterate all - toArray() + loop 32.954 ms/op 30.681 ms/op 1.07
Tree 40 250000 iterate all - get(i) 110.96 ms/op 108.84 ms/op 1.02
MutableVector 250000 create 16.364 ms/op 15.099 ms/op 1.08
MutableVector 250000 get(125000) 13.209 ns/op 13.124 ns/op 1.01
MutableVector 250000 set(125000) 676.96 ns/op 609.28 ns/op 1.11
MutableVector 250000 toArray() 6.7106 ms/op 6.1955 ms/op 1.08
MutableVector 250000 iterate all - toArray() + loop 6.7376 ms/op 7.2551 ms/op 0.93
MutableVector 250000 iterate all - get(i) 3.7232 ms/op 3.5261 ms/op 1.06
Array 250000 create 7.3192 ms/op 6.0561 ms/op 1.21
Array 250000 clone - spread 2.1669 ms/op 2.1183 ms/op 1.02
Array 250000 get(125000) 0.89200 ns/op 0.90700 ns/op 0.98
Array 250000 set(125000) 0.91300 ns/op 0.90300 ns/op 1.01
Array 250000 iterate all - loop 167.93 us/op 167.74 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 63.994 us/op 60.525 us/op 1.06
effectiveBalanceIncrements clone MutableVector 300000 572.00 ns/op 555.00 ns/op 1.03
effectiveBalanceIncrements rw all Uint8Array 300000 252.77 us/op 252.85 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 175.71 ms/op 163.19 ms/op 1.08
phase0 afterProcessEpoch - 250000 vs - 7PWei 186.00 ms/op 184.77 ms/op 1.01
phase0 beforeProcessEpoch - 250000 vs - 7PWei 78.814 ms/op 72.265 ms/op 1.09
altair processEpoch - mainnet_e81889 597.79 ms/op 492.80 ms/op 1.21
mainnet_e81889 - altair beforeProcessEpoch 154.12 ms/op 132.55 ms/op 1.16
mainnet_e81889 - altair processJustificationAndFinalization 21.290 us/op 21.737 us/op 0.98
mainnet_e81889 - altair processInactivityUpdates 11.784 ms/op 11.510 ms/op 1.02
mainnet_e81889 - altair processRewardsAndPenalties 94.393 ms/op 89.430 ms/op 1.06
mainnet_e81889 - altair processRegistryUpdates 4.1010 us/op 2.9230 us/op 1.40
mainnet_e81889 - altair processSlashings 880.00 ns/op 724.00 ns/op 1.22
mainnet_e81889 - altair processEth1DataReset 970.00 ns/op 792.00 ns/op 1.22
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4860 ms/op 2.3087 ms/op 1.08
mainnet_e81889 - altair processSlashingsReset 5.9970 us/op 6.1460 us/op 0.98
mainnet_e81889 - altair processRandaoMixesReset 7.6300 us/op 5.4640 us/op 1.40
mainnet_e81889 - altair processHistoricalRootsUpdate 972.00 ns/op 779.00 ns/op 1.25
mainnet_e81889 - altair processParticipationFlagUpdates 3.3080 us/op 3.0260 us/op 1.09
mainnet_e81889 - altair processSyncCommitteeUpdates 777.00 ns/op 746.00 ns/op 1.04
mainnet_e81889 - altair afterProcessEpoch 195.16 ms/op 193.90 ms/op 1.01
phase0 processEpoch - mainnet_e58758 543.28 ms/op 517.34 ms/op 1.05
mainnet_e58758 - phase0 beforeProcessEpoch 261.26 ms/op 222.76 ms/op 1.17
mainnet_e58758 - phase0 processJustificationAndFinalization 20.849 us/op 26.668 us/op 0.78
mainnet_e58758 - phase0 processRewardsAndPenalties 79.492 ms/op 132.01 ms/op 0.60
mainnet_e58758 - phase0 processRegistryUpdates 9.9720 us/op 9.9380 us/op 1.00
mainnet_e58758 - phase0 processSlashings 820.00 ns/op 678.00 ns/op 1.21
mainnet_e58758 - phase0 processEth1DataReset 916.00 ns/op 722.00 ns/op 1.27
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.1892 ms/op 1.9230 ms/op 1.14
mainnet_e58758 - phase0 processSlashingsReset 4.8100 us/op 4.7330 us/op 1.02
mainnet_e58758 - phase0 processRandaoMixesReset 6.8610 us/op 5.2880 us/op 1.30
mainnet_e58758 - phase0 processHistoricalRootsUpdate 881.00 ns/op 739.00 ns/op 1.19
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.8250 us/op 4.6140 us/op 1.05
mainnet_e58758 - phase0 afterProcessEpoch 167.87 ms/op 159.59 ms/op 1.05
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.8044 ms/op 2.6549 ms/op 1.06
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.2738 ms/op 3.4107 ms/op 0.96
altair processInactivityUpdates - 250000 normalcase 44.061 ms/op 37.709 ms/op 1.17
altair processInactivityUpdates - 250000 worstcase 50.913 ms/op 46.451 ms/op 1.10
phase0 processRegistryUpdates - 250000 normalcase 7.9150 us/op 12.243 us/op 0.65
phase0 processRegistryUpdates - 250000 badcase_full_deposits 443.41 us/op 572.40 us/op 0.77
phase0 processRegistryUpdates - 250000 worstcase 0.5 220.78 ms/op 207.33 ms/op 1.06
altair processRewardsAndPenalties - 250000 normalcase 127.19 ms/op 130.69 ms/op 0.97
altair processRewardsAndPenalties - 250000 worstcase 87.311 ms/op 81.183 ms/op 1.08
phase0 getAttestationDeltas - 250000 normalcase 13.235 ms/op 12.143 ms/op 1.09
phase0 getAttestationDeltas - 250000 worstcase 13.401 ms/op 12.249 ms/op 1.09
phase0 processSlashings - 250000 worstcase 5.8876 ms/op 5.9141 ms/op 1.00
altair processSyncCommitteeUpdates - 250000 269.75 ms/op 261.94 ms/op 1.03
BeaconState.hashTreeRoot - No change 420.00 ns/op 456.00 ns/op 0.92
BeaconState.hashTreeRoot - 1 full validator 61.392 us/op 55.647 us/op 1.10
BeaconState.hashTreeRoot - 32 full validator 751.18 us/op 652.61 us/op 1.15
BeaconState.hashTreeRoot - 512 full validator 6.6374 ms/op 5.9322 ms/op 1.12
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 79.397 us/op 75.505 us/op 1.05
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2053 ms/op 1.3102 ms/op 0.92
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 15.852 ms/op 15.370 ms/op 1.03
BeaconState.hashTreeRoot - 1 balances 61.440 us/op 58.514 us/op 1.05
BeaconState.hashTreeRoot - 32 balances 578.29 us/op 580.06 us/op 1.00
BeaconState.hashTreeRoot - 512 balances 6.0012 ms/op 5.7850 ms/op 1.04
BeaconState.hashTreeRoot - 250000 balances 97.601 ms/op 94.364 ms/op 1.03
aggregationBits - 2048 els - zipIndexesInBitList 29.662 us/op 28.554 us/op 1.04
regular array get 100000 times 67.316 us/op 67.370 us/op 1.00
wrappedArray get 100000 times 67.385 us/op 67.382 us/op 1.00
arrayWithProxy get 100000 times 27.749 ms/op 28.051 ms/op 0.99
ssz.Root.equals 1.0750 us/op 1.0790 us/op 1.00
byteArrayEquals 1.0630 us/op 1.0610 us/op 1.00
shuffle list - 16384 els 11.298 ms/op 12.572 ms/op 0.90
shuffle list - 250000 els 166.10 ms/op 165.54 ms/op 1.00
processSlot - 1 slots 12.583 us/op 11.511 us/op 1.09
processSlot - 32 slots 1.7586 ms/op 1.6784 ms/op 1.05
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 344.95 us/op 332.36 us/op 1.04
getCommitteeAssignments - req 1 vs - 250000 vc 5.5877 ms/op 5.6217 ms/op 0.99
getCommitteeAssignments - req 100 vs - 250000 vc 7.5630 ms/op 7.5874 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 8.1879 ms/op 8.1159 ms/op 1.01
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 9.7100 ns/op 9.6500 ns/op 1.01
state getBlockRootAtSlot - 250000 vs - 7PWei 1.2201 us/op 1.1349 us/op 1.08
computeProposers - vc 250000 16.282 ms/op 15.687 ms/op 1.04
computeEpochShuffling - vc 250000 172.16 ms/op 170.15 ms/op 1.01
getNextSyncCommittee - vc 250000 270.34 ms/op 260.60 ms/op 1.04

by benchmarkbot/action

@dapplion dapplion marked this pull request as ready for review December 14, 2022 17:02
@dapplion dapplion requested a review from a team as a code owner December 14, 2022 17:02
/** From https://notes.ethereum.org/@vbuterin/extended_light_client_protocol#Optimistic-head-determining-function */
const SAFETY_THRESHOLD_FACTOR = 2;

export function sumBits(bits: BitArray): number {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is defined in 3 different places across the code base now. Here, in the beacon node package, and in the light client src. Might be good to have it only in the util package.

Copy link
Contributor Author

@dapplion dapplion Dec 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such a tinny function, which is just an alias for readability on the SSZ method. I think it's better to have 2 extra lines than a random function in the utils package.

// If update is not finalized finalizedHeaderSlot does not matter (see is_better_update), so setting
// to zero to set it some number.
finalizedHeaderSlot: attestedData.isFinalized
? computeStartSlotAtEpoch(attestedData.finalizedCheckpoint.epoch)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it safe to assume the finalised header slot will always be the first slot at the epoch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this very specific case, IMO yes. That finalizedHeaderSlot is used only to break ties in the is_better_update function. If the finalized checkpoint advances is extremely unlikely that the referenced block's slot does not increase. Even if it didn't the consequence is just persisting a slightly different update under some circumstances

}

// Sanity check that slots are in correct order
if (update.signatureSlot <= update.attestedHeader.slot) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec also include the current_slot >= update.signature_slot assertion. Is it a conscious decision to not include that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, to remove the dependency on the clock from this function. That check is done on the caller of this function

};
const lightClient = new LightclientSpec(config, lightClientOpts, testcase.bootstrap);

// fromHex(testcase.meta.trusted_block_root)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment can be removed.

Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@g11tech g11tech merged commit b1549a6 into unstable Dec 19, 2022
@g11tech g11tech deleted the dapplion/lightclient-spec-tests branch December 19, 2022 07:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants